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Recent  Developments  in  the  Complexity  of  Combinatorial  Algorithms 


Robert  Endre  Tar j  an 
Computer  Science  Department 
Stanford  University 
Stanford,  California  94305 


1.  Introduction. 

In  a  1978  paper  (Tarjan  [1978]),  the  author  surveyed  the  then-current  state 
of  knowledge  concerning  combinatorial  algorithms.  Since  the  time  that  paper  was 
published,  researchers  in  the  area  have  obtained  several  major  new  results.  These 
include  a  sequence  of  more-and-more  efficient  algorithms  for  matrix  muliiplicfr 
tion,  a  similar  sequence  of  algorithms  for  maximum  network  flow,  a  polynomiai- 
time  algorithm  for  linear  programming,  and  a  number  of  new  approaches  to  graph 
isomorphism.  In  this  paper  we  shall  examine  these  recent  advances,  attempt  to 
access  their  significance,  and  suggest  directions  for  future  research.  The  paper  in¬ 
cludes  an  examination  of  recent  work  by  the  author  and  his  students  on  dynamic 
dictionaries,  maximum  flow  problems,  and  related  topics.  Much  of  the  work  we 
shall  describe  is  still  unpublished. 

Before  beginning  this  survey,  it  is  useful  to  review  the  general  framework 
in  which  we  shall  view  combinatorial  algorithms  and  their  complexity.  We  are 
interested  in  sequential  algorithms;  that  is,  in  algorithms  that  perform  only  one 
step  at  a  time.  As  a  computer  model,  we  use  either  a  random-access  machine  with 
uniform  cost  measure  (Aho,  Hopcroft,  and  Ullman  [1974]),  or  a  pointer  machine 
(Tarjan  [1579]).  With  either  machine  model  we  can  perform  a  single  arithmetic 
or  logical  operation  in  constant  time,  and  we  can  store  or  retrieve  a  single  pieco 
of  information  in  constant  time. 

Tfce  difference  between  random  access  machines  and  pointer  machines  lies  in 
their  memory  organisation.  The  memory  of  a  Fandom  access  machine  consists  of 
a  one-dimensional  array  of  cells,  each  capable  of  holding  a  single  piece  of  infor¬ 
mation  (such  as  a  real  number).  The  memory  of  a  pointer  machine  is  a  linked 
structure  consisting  of  a  collection  of  nodes  linked  by  pointers;  each  node  consists 
of  a  fixed  finite  number  of  fields,  some  of  which  are  designated  as  containing 
pointers  to  other  nodes.  See  Figure  1.  Random  access  machines  seem  inherently 
more  powerful  than  pointer  machines,  since  we  can  perform  arithmetic  on  array 
addresses  but  not  on  pointers.  However,  this  difference  in  power  amounts  to  at 
most  a  factor  of  log  n  in  running  time,  and  in  fact  the  algorithms  we  shall  discuss 
have  the  same  asymptotic  running  times  on  either  random-access  machines  or 
pointer  machines. 
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(Figure  1] 

As  a  complexity  measure  we  use  the  worst-case  running  time  of  an  algorithm 
as  a  function  of  the  sise  of  its  input.  We  shall  ignore  constant  factors.  For  graph 
problems  we  shall  use  n,  the  number  of  vertices,  and  m,  the  number  of  edges,  to 
measure  the  input  sise.  For  algorithms  that  manipulate  real  numbers,  measuring 
input  sise  is  more  problematic.  The  issue  is  whether  we  should  allow  rational 
numbers  of  arbitrary  precision  (or  even  irrational  numbers)  as  input,  and  count 
any  arithmetic  operation  as  a  single  step,  or  whether  we  should  measure  the  sise 
of  real  numbers  by  the  number  of  bits  needed  to  represent  them,  and  count  each 
bit  operation  as  a  single  step.  The  former  approach  is  truer  to  the  spirit  of  al¬ 
gebraic  complexity,  and  we  shall  adopt  it  in  the  case  cf  matrix  multiplication 
and  maximum  network  flow.  The  latter  approach  is  more  reasonable  if  we  are 
interested  in  issues  of  NP-comp!eteness,  and  we  shall  adopt  it  in  the  case  of  linear 
programming. 

The  paper  comprises  7  sections.  Section  2  examines  the  recently  discovered 
ellipsoid  method  for  linear  programming  and  its  implications  for  combinatorial 
complexity.  Section  3  discusses  graph  isomorphism.  Section  4  surreys  new  work 
on  matrix  multiplication.  Section  5  examines  recent  results  on  dynamic  dic¬ 
tionaries,  and  Section  6  discusses  network  flow.  Section  7  contains  conclusions 
and  remarks  about  promising  directions  for  future  research. 
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2.  Linear  Programming* 

Garey  and  Johnson's  book  on  NP-completeness  (Garey  and  Johnson  {1979]) 
concludes  with  a  list  of  problems  whose  status  with  respect  to  NP-completeness 
was  unknown  at  the  time  the  book  was  published.  A  prominent  problem  on 
this  list  is  //near  programming,  which  can  be  defined  as  follows:  given  a  set  of 
linear  inequalities  a ,-x  <  6,-  for  1  <  t  <  m,  find  a  vector  z  that  satisfies  these 

inequalities  and  maximizes  cz  =  CiXj  -i - \~cKzn.  Here  x  —  (zi,...,zR)  is  a 

vector  of  n  variables,  each  os-  is  a  rector  of  n  real  numbers,  and  the  &»’ s  and  cy’s 
are  real  numbers. 

There  is  a  well-known  and  empirically  efficient  algorithm  for  linear  program-  f 

ming,  Dantzig’s  simplex  method  (Dantzig  [JS51]),  but  this  algorithm  runs  in  ex¬ 
ponential  time  on  certain  sets  of  input  data  (Klee  and  Minty  [1972]).  However, 

Xhachiyan  [1979]  managed  to  prow  that  a  completely  different  method  based 
upon  ideas  of  Shor  [1970, 1977]  solves  the  linear  programming  problem  in  polyno¬ 
mial  time.  The  algorithm,  which  we  shall  call  the  ellipsoid  method,  is  surprisingly 
simple,  and  in  retrospect  it  is  a  wonder  that  it  was  not  discovered  many  years 
previously. 

The  key  idea  of  the  method  is  to  use  binary  search,  maintaining  an  ellipsoid 
that  bounds  the  set  of  solutions.  We  shall  assume  that  the  a,’s,  5,’s  and  e/s 
consist  of  integers,  and  that  the  size  of  the  input  is  measured  by  the  number  of 
bits  l  required  to  write  down  all  these  integers  in  binary.  We  note  first  that  if  the 
function  cz  has  a  finite  maximum,  then  the  value  of  x  achieving  the  maximum 
has  a  number  of  bits  polynomial  in  l.  If  we  add  to  our  set  of  inequalities  an 
inequality  cz  >  a,  where  a  is  a  parameter,  we  can  use  binary  search  on  a  to 
determine  the  maximum  value  of  o  for  which  a  feasible  solution  exists. 

Thus  we  have  reduced  our  original  optimization  problem  to  the  problem  of 
solving  a  polynomial  number  of  feasibility  problems  of  the  following  form:  given  a 
set  of  inequalities  a,-z  <  for  1  <  t  <  m,  is  there  some  value  of  z  that  satisfies 
all  the  inequalities?  By  means  of  a  second  transformation  (perturbing  each  ft,-  by 
a  sufficiently  small  value)  we  can  make  all  the  inequalities  strict. 

Now  we  come  to  the  heart  of  the  matter.  In  order  to  search  for  feasible  solu¬ 
tions,  we  construct  an  ellipsoid  guaranteed  to  contain  at  least  a  certain  volume  of 
solutions  if  there  are  any  solutions  at  all.  It  suffices  to  choose  a  sphere  centered 
at  the  origin  with  sufficiently  large  (but  polynomially  bounded)  radius.  We  then 
repeatedly  reduce  the  size  of  the  bounding  ellipsoid  until  either  we  find  a  solution 
or  the  bounding  ellipsoid  is  so  small  that  it  can't  contain  any  solutions,  and  thus 
there  are  po  solutions. 

To  carry  out  a  general  step  of  this  reduction  process,  we  test  the  center  z 
of  the  ellipsoid  for  feasibility.  If  x  satisfies  all  the  inequalities,  we  are  done.  If 
not,  we  find  a  violated  inequality  o,-x  >  We  then  construct  a  new  ellipsoid 
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containing  those  points  in  the  old  ellipsoid  that  satisfy  o,z  <  See  Figure  2. 
The  crucial  point  is  that  the  volume  of  the  new  ellipsoid  is  smaller  by  a  constant 
factor  (depending  polynomially  on  n)  than  the  volume  of  the  old  ellipsoid;  thus 
after  a  polynomial  number  of  step3  either  we  find  a  feasible  solution  or  we  can 
terminate  the  process  and  ueciare  that  no  solutions  exist.  For  further  details  of 
the  algorithm,  see  Gacs  and  Loras*  [1979J  o?  Asprall  and  Stone  {1979}. 

{Figure  2} 

Kkachiyan’s  discovery  has  led  to  an  explosion  of  research  on  related  issues. 
Of  primary  interest  are  the  questions,  “Is  the  algorithm  efficient  (or  can  it  be  made 
so)?”  and  “What  implications  does  the  algorithm  have  for  other  combinatorial 
problem?,?’  As  a  tentative  answer  to  the  first  question,  it  seems  that  the  ellipsoid 
method  is  not  competitive  in  practice  with  the  simplex  method,  although  this 
matter  deserves  further  study. 

The  theoretical  implications  of  the  ellipsoid  method  are  more  interesting. 
The  algorithm  doe3  not  depend  heavily  on  linearity,  only  on  the  convexity  of 
the  solution  space.  Indeed,  Koilov,  Tarasov,  and  Khaciyan  {1979}  hare  extended 
the  algorithm  to  convex  quadratic  programming.  On  the  other  hand,  the  linear 
case  is  the  most  interesting  from  the  point  of  view  of  combinatorics.  A  standard 
technique  of  operations  researchers  is  to  use  linear  programming  to  attack  integer 
programming  problems.  The  idea  is  to  define  an  integer  programming  problem 
?s  a  iinear  programming  pioblem  with  a  large  (possibly  exponential)  number 
of  inequalities,  in  such  a  way  the  Inequalities  defining  the  problem  are  easy  to 
generate.  Karp  and  Papadimitriou  {1979}  have  proved  a  negative  result  for  this 
approach,  any  NP-complete  combinatorial  optimisation  problem  cannot  be  poiy- 
nomially  characterised  by  a  set  of  linear  inequalities,  unless  N?  =  co-NP.  By 
“polynomially  characterised"  we  mean  that  the  set  of  inequalities  is  in  NP;  that 
is,  given  an  inequality  that  is  in  the  set  we  can  construct  a  polynomial-length 
proof  of  this  fact.  Since  it  is  unlikely  that  NP  =  co-NP,  this  means  that  any 
combinatorial  optimisation  problem  that  can  be  polynomially  characterised  by 
inequalities  is  unlikely  tc  be  NP-complete. 

In  order  to  run  the  ellipsoid  method,  we  do  not  need  an  explicit  listing  of 
ail  the  inequalities  defining  the  problem  but  only  a  way  to  te3t  whether  a  point 
is  feasible  and  to  generate  a  violated  inequality  if  it  is  not.  Suppose  we  have  a 
combinatorial  opimisation  problem  for  which  it  i?.  possible  to  test  feasibility  and 
generate  a  violated  inequality  in  polynomial  time.  Th~n  by  means  of  Khacbiyan’g 
algorithm  we  can  solve  the  combinatorial  optimisation  problem  in  polynomial 
time  (if  certain  other  weak  conditions  are  satisfied).  Karp  and  Papadimitrsou 
and  independently  Grotschei,  LoviUs,  and  Schrijver  [1980}  made  this  observation, 


wuich  implies  that  NP-complete  problems  are  unlikely  to  have  polynomial  lime 
generators  of  violated  inequalities.  The  result  also  means  that  the  ellipsoid  method 
is  in  some  sense  a  universal  method  for  combinatorial  optimisation  problems,  and 
it  can  be  used  to  generate  sew  fast  algorithms.  Fof  instance,  Grotschel,  Lovass, 
and  Schnjver  have  derived  a  polynomial-time  algorithm  for  vertex  packing  in 
perfect  graphs.  This  area  appears  to  be  ripe  for  farther  -work.  For  an  excellent 
non-technical  discussion  of  Khacfciyan’s  algorithm  and  its  implication  for  com¬ 
binatorial  optimisation,  see  Lovass  (ISSOj. 
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3-  O&ph  Isomorphism. 

Another  cd  Garey  and  Johnson’s  open  problems  is  graph  isomorphism:  given 
two  undirected  graphs  G\  and  G2f  determine  whether  there  is  a  one-to-one  map¬ 
ping  of  the  vertices  of  Gi  onto  the  vertices  of  G2  that  preserves  adjacency.  This 
problem  has  a  number  of  applications,  especially  in  the  cataloguing  of  chemical 
?ic-lecules.  Although  no  one  has  yet  discovered  a  polynomial-time  algorithm  for 
graph  isomorphism,  much  progress  has  been  made  recently. 

Most  of  the  early  algorithms  for  graph  isomorphism  (see  for  instance  Comeil 
and  Gotlieb  (1970),  Read  and  Comeil  (1977j)  combine  backtrack  search  with 
3  partition  refinement  method  to  reduce  the  site  of  the  search  space.  Such  a 
method  is  very  efficient  in  practice  but  requires  exponential  time  on  tome  highly 
symmetric  graphs,  because  the  refinement  scheme  becomes  useless.  For  various 
special  classes  of  graphs,  efficient  algorithms  are  known.  These  classes  include 
trees  (Aho,  Hopcroft,  and  Ullman  (1974]).  pianar  graphs  (Hopcroft  and  Tarjan 
(1972],  Hopcroft  and  Wong  ]1974]),  series-parallel  graphs  (Valdes,  Tarjan,  and 
Lawler  (1979]),  and  interval  graphs  (Cclbourn  and  Booth  (1979]),  for  which  linear- 
time  algorithms  exist.  The  tree  isomorphism  algorithm  depends  upon  clever  use 
o?  lexicographic  sorting.  The  algorithms  for  series-parallel  graphs  and  interval 
graphs  arc  straightforward  extensions  of  the  algorithm  for  trees.  The  algorithm 
for  planar  graphs  combines  the  tree  isomorphism  algorithm  with  a  linear-time 
decomposition  into  triconnected  components  (Hopcroft  and  Tarjan  (1973]),  and 
a  reduction  method  for  triconnected  embedded  planar  graphs.  The  method  is 
fast  for  three  reasons:  the  relationship  among  triconnected  components  can  be 
represented  by  a  decomposition  tree,  a  triconnected  planar  graph  has  only  two 
planar  embeddings,  and  there  is  a  liDear-time  algorithm  for  embedding  a  planar 
graph  (Hopcroft  and  Tarjan  (1974]).  Recently  Lichtenstein  (1980]  has  discovered 
a  polynomial-time  isomorphism  algorithm  for  graphs  embeddable  in  a  projective 
plane,  and  Miller  (1980]  and  Filotti  and  Mayer  [1980]  have  found  a  polynomial¬ 
time  isomorphism  algorithm  for  graphs  of  any  fixed  genus.  These  methods  com¬ 
bine  fast  embedding  algorithms  (Filotti,  Miller,  and  Reif  {1979])  with  a  careful 
analysis  of  the  ways  a  g~aph  can  be  embedded  in  a  surface  of  the  appropriate 
genus. 

Other  researchers  have  attempted  to  use  degree  constraints  or  symmetry 
properties  to  aid  in  testing  isomorphism.  Miller  (1977)  showed  that  isomorphism 
is  in  co-NP  for  arc-transitive  trident  graphs;  that  is,  if  two  such  graphs  are 
non-isomorphic,  thre  is  a  polynomial-1  sngth  proof  of  this  fact.  Lipton  (1980] 
discovered  an  -time  isomorphism  algorithm  for  arc-transitive  trivalent 

graphs,  and  Babai  (1980]  found  an  -time  algorithm  for  strongly  regular 

graphs.  The  major  breakthrough  was  Babai*?  discovery  (Babai  [1979])  of  a  random 
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polynomial-time  algorithm  for  rertez-colored  graphs  with  bounded  color  multi¬ 
plicities.  Babai’s  work  is  important  much  more  for  his  techniauex  than  for  the 
specific  results  to  be  obtained.  He  was  able  to  use  properties  of  the  automorphism 
group  o?  a  graph,  describing  this  group  by  means  of  a  tower  of  subgroups,  in 
such  a  way  that  one  could  quickly  construct  the  tower.  This  work  was  extended 
by  Hoffman  {1930},  who  found  a  random  R^°*a'-iime  algorithm  for  a  class  of 
graphs  called  "cone  graphs* .  Furst,  Eopcroft,  and  Luks  (private  communication) 
made  Babai’s  methods  deterministic  and  generalized  Hoffman's  algorithm  to  test 
isomorphism  of  triralent  graphs  deterministically  in  n^P°*n)  time.  Luks  {1980] 
reduced  the  running  time  of  the  algorithm  for  iiiraient  graphs  to  polynomial  and 
found  an  n^lo»  n'-time  algorithm  for  graphs  of  any  fixed  Talencs. 

Extrapolating  this  work,  it  seems  likely  that  a  polynomial-time  isomorphism 
algorithm  for  graphs  of  aay  fixed  degree  will  soon  be  discovered.  Whether  this 
will  lead  to  a  polynomial-time  algorithm  for  the  general  problem  is  less  clear. 
Much  cf  the  recent  work  on  isomorphism  depends  heavily  on  properties  of  finite 
groups,  and  a  complete  solution  may  require  new  results  in  group  theory  or  at 
least  ingenious  use  of  old  ones. 
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4.  Matrix  Multiplication. 

Not  all  research  in  combinatorial  complexity  has  been  derated  to  finding 
polynomial-time  algorithms  for  problems  not  known  to  hare  them.  Much  effort 
has  also  been  devoted  to  finding  faster  algorithms  for  problems  already  known 
to  be  solvable  in  polynomial  time.  For  example,  let  ns  examine  recent  progress 
on  matrix  multiplication.  See  Table  1.  The  classical  algorithm  for  this  problem 
multiplies  two  nXn  matrices  in  0{n5)  time.  Strassen  [1969]  discovered  a  way  to 
multiply  two  2X2  matrices  with  only  seven  multiplications,  and  tiling  this  obser¬ 
vation  constructed  a  recursive  algorithm  to  multiply  n  X  n  matrices  in  0(ftIc**T) 
time. 

[Table  1] 

After  a  hiatus  of  nine  yean,  Pan  [1978}  found  a  slightly  faster  (0{n=-T*5) 
rs.  0(n2  801))  method.  Pan  used  an  ingenious  and  complicated  technique  called 
“aggregating,  uniting,  and  cancelling.*  A  key  point  in  Pan’s  approach  was  to 
use  fairly  large  sise  matrices  (n  =  70)  as  the  basis  for  the  recursion,  rathr  than 
very  small  matrices  (n  =  2  or  3  or  4).  Shortly  after  Pan’s  result  appeared,  Bini, 
Capovani,  Romani,  and  Lotti  [1979]  introduced  a  notion  of  "approximate*  matrix 
multiplication,  and  produced  an  0(n2  7c)  -time  algorithm  to  multiply  matrices  in 
this  approximate  sense.  Schonhage  [1979]  showed  that  any  approximate  matrix 
multiplication  algorithm  can  be  converted  into  an  exact  one,  and  he  further 
showed  that  any  method  for  multiplying  sparse  matrices  can  be  converted  into  a 
method  to  multiply  dense  ones.  The  culmination  of  Schonhage’s  advances  was  an 
O(n2.soss)  .time  algorithm.  Subsequent  work  by  Fan,  Schonhage,  and  Wincgrad, 
using  a  combination  of  these  techniques,  has  produced  a  sequence  of  fastor-and- 
f aster  algorithms;  the  best  bound  currently  claims!  is  0(n2-4*+},  due  to  Pan 
(private  communication).  Pan  has  recently  written  a  long  paper  (Pan  [1930])  sur¬ 
veying  these  developments.  It  may  be  possible  to  multiply  matrices  in  0(n?+*) 
time  for  any  positive  e;  certainly  the  recent  results  lead  in  this  direction. 


5.  Dynamic  Dictionaries. 

Fast  algorithms  require  the  use  of  appropriate  data  structures,  some  of  which 
are  quite  complicated.  In  this  section  we  shall  examine  efficient  wajs  to  represent 
one  important  type  of  data  structure,  called  a  dynamic  dictionary.  A  dynamic 
dictionary  consists  of  a  collection  items,  each  with  an  associated  key.  We  assume 
that  the  keys  are  totally  ordered  and  can  be  compared;  we  further  assume  that 
no  two  keys  are  the  same.  We  are  interested  in  performing  the  following  kinds  of 
operations  cn  dynamic  dictionaries: 

(a)  Given  a  key,  access  the  item  (if  any)  with  this  key. 

(b)  Insert  a  new  item  in  the  dictionary. 

(c)  Delete  a  given  item  from  the  dictionary. 

(d)  Merge  two  dictionaries  into  a  single  dictionary. 

(e)  Concatenate  two  dictionaries,  such  that  al!  keys  in  one  dictionary  are  smaller 
than  all  keys  in  the  cthei. 

(f)  Split  a  dictionary  on  a  given  key  into  a  dictionary  containing  all  keys  no 
larger  than  the  given  key  and  a  dictionary  containing  all  keys  larger  than  the 
given  key. 

Dynamic  dictionaries  have  widespread  uses  in  computer  science;  see  Knuth 
(1973).  There  are  a  number  of  ways  to  represent  such  dictionaries.  If  only  accesses 
and  insertions  are  to  be  performed,  a  hash  table  (Knuth  [1973])  can  be  used.  Ha*b 
tables  allow  accesses  and  insertions  in  0(1)  time  on  the  average,  although  the 
worst-case  time  is  0(n),  where  n  is  the  number  of  items  in  the  dictionary.  Hash 
tables  tan  be  adapted  to  allow  efficient  deletion,  but  not  merging,  concatena¬ 
tion,  or  splitting;  such  tables  do  not  maintain  the  ordering  information  needed  to 
efficiently  merge,  concatenate,  or  split.  A  balanced  tree  structure,  such  as  a  2  —  3 
tree  (Ahc,  Hopcroft,  and  Uilman  [1974])  or  more  generally  a  B-tree  (Bayer  and 
McCreight  [1972]),  a  height-balanced  tree  (Knuth  [1973]),  or  a  weight-balanced 
tree  (Reingold,  Nievergelt,  and  Deo  [1974]),  is  appropriate  if  such  operations  are 
tc  be  performed.  Such  a  structure  allows  access,  insertion,  deletion,  concatena¬ 
tion,  and  splitting  in  O(logn)  time  (see  Aho,  Hopcroft,  and  Uilman  [1974]  for 
instance). 

There  are  a  number  of  recent  results  on  dynamic  dictionaries.  Brown  and 
Tarjan  [1979]  showed  how  to  merge  two  dictionaries  represented  as  height-balanced 
trees  in  0(mlog  %)  time,  if  the  smaller  dictionary  has  m  items  and  tb  larger  one 
has  n  items.  This  result  also  holds  for  B-trees  and  for  weight-balanced  trees. 
Brown  and  Tarjan  [1980]  showed  how  to  maintain  “fingers”  into  2—3  trees  so  that 
access  is  very  fast  on  the  vicinity  of  a  finger.  Their  proposed  structure  supports 
fast  access,  finger  creation,  insertion,  and  deletion,  as  long  as  the  insertions  and 
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deletions  occur  in  separate  parts  of  the  tree.  Huddleston  (private  communication), 
Maier  and  Salveter  {1979),  and  Mehlhorn  [1979b]  independently  used  less  balanced 
versions  of  B-trees  to  extend  Brown  and  Tarjan's  results  so  that  arbitrary  inser¬ 
tions  and  deletions  are  fast. 

An  interesting  question  is  what  happens  if  we  want  to  bias  the  dictionary  so 
that  certain  items  are  easier  to  access  than  others.  This  is  desirable,  for  instance, 
in  keyword  tables  for  compilers  and  in  language  dictionaries,  where  some  words 
are  accessed  much  more  often  than  others.  To  study  this  qv^rtion  we  nssume 
that  each  item  c  has  a  weight  w,-,  end  that  we  wish  to  minimise  the  sum  of  the 
weighted  access  times  £"=i  wiU,  where  £,•  is  the  access  time  of  item  s'.  Knuth 
(1971)  and  Hu  and  Tucker  [1971]  have  proposed  efficient  algorithms  to  construct 
optimum  binary  search  trees ;  Knuth’s  algorithm  requires  0 (n2)  time  but  allows 
items  to  be  stored  in  internal  nodes  of  the  tree;  Hu  and  Tucker’s  algorithm  uses 
O(nlogn)  time  but  requires  that  all  the  items  be  in  external  tree  nodes.  Garsia 
and  Wachs  [1977]  have  given  an  interesting  variant  of  the  Hu- Tucker  algorithm. 

Optimum  binary  search  trees  are  not  suitable  if  insertions  and  deletions 
are  to  be  performed,  because  they  require  too  much  time  to  update.  Mehlhcrn 
[1978,  1979a]  has  investigated  the  question  of  dynamically  maintaining  an  almcsv 
optimum  tree.  An  entropy  argument  shows  that  the  sum  of  the  weighted  access 
times  in  an  optimum  tree  is  bounded  below  by  a  constant  times  w,-log 
where  w  —  £"=1  to,-;  thus  the  goal  is  to  maintain  a  tree  in  which  each  item  i 
has  0(1  -f  log  access  time.  Mehlhorn  has  described  a  complicated  version  of 
weight-balanced  trees  with  the  following  properties: 

(a)  0(1  -f  log  jjfc)  time  to  access,  insert,  or  delete  item  s'; 

(b)  0(1  -f-  log  time  to  change  the  weight  of  item  »  from  so,-  to 

where  w  is  the  total  weight  before  the  change  and  vj1  is  the  total  weight  after 

the  change. 

Bent,  Sleator,  and  Tarjan  [1980]  have  found  a  way  to  implement  dynamic  dic¬ 
tionaries  that  is  not  only  much  simpler  than  Mehlhorn’s  but  allows  fast  concatena¬ 
tion  and  splitting;  specifically,  0(1  -f-  log$)  time  to  concatenate  dictionaries  of 
total  weights  tv  and  w1  with  w  >  w',  and  0(1  +  log  time  to  split  a  dictionary 
at  item  t.  The  data  structure  resembles  a  2  —  3  tree.  Each  node  in  the  tree  has 
between  zero  and  three  children.  Certain  nodes  contain  items;  others  are  non¬ 
item  nodes  (an  item  node  contains  exactly  one  item).  A  symmetric-order  traversal 
of  the  tree  visits  the  items  in  order  by  key.  An  item  node  has  a  left  son  and 
a  right  son,  either  or  both  of  which  can  be  missing;  a  non-item  node  has  either 
two  or  three  children.  In  addition,  each  node  has  a  level,  defined  as  follows:  the 
level  of  a  node  containing  item  i  is  [log«y,J;  the  level  of  a  non-item  node  is  one 
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greater  than  the  minimum  of  the  levels  of  its  children.  We  impose  the  additional 
requirement  that  the  level  of  a  node  be  strictly  greater  than  the  levels  of  all  its 
children;  thus  all  children  of  a  non-item  node  have  the  same  level.  Figure  3  gives 
an  example  of  such  a  tree. 

[Figure  3] 

It  is  not  hard  to  implement  access,  insertion,  deletion,  concatenation,  split- 
ting,  and  weight  change  on  such  trees.  Analysing  the  efficiency  of  these  operations, 
however,  requires  a  clever  accounting  argment.  The  data  structure  has  not  only 
the  obvious  applications  but  also  a  number  of  not-so-obvious  ones,  as  we  shall  see 
in  the  next  section. 
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8.  Maximum  Network  Flow* 

An  important  special  case  of  linear  programming  is  the  maximum  network 
flow  problem:  given  a  directed  graph  G  =  (V”,  B),  two  distinguished  vertices,  a 
source  s  and  a  sink  t,  and  a  non-negative  capacity  c(t;  on  each  edge  e,  find  a 
flow  of  maximum  value  frcm  <t  to  t.  A  Bow  is  defined  by  a  value  /(e)  on  each 
edge  e,  such  that  0  <  f(c)  <  c(e);  tor  every  vertex  u  except  the  source  and  sink, 
the  total  flow  on  edges  entering  v  must  equal  the  total  flow  on  edges  leaving  v. 
The  value  of  the  Sow  is  the  total  flow  on  edges  leaving  the  source  (or  equivalently 
on  edges  entering  the  sink). 

Ford  and  Fulkerson  {1962]  were  the  first  to  study  this  problem.  They  proved 
the  famous  max-Bow  min-cut  theorem ,  which  states  that  the  value  of  a  maximum 
flow  equals  the  capacity  of  a  minimum  cut.  (A  cut  is  a  partition  X,X  of  the 
vertices  such  that  s  6  X  and  t  £  X;  the  capacity  of  the  cut  is  the  total  capacity 
of  edges  leaving  X  and  entering  X.)  They  proved  this  theorem  by  means  of  an 
augmenting  path  method  which,  given  a  flow,  attempts  to  find  a  path  from  s  to  t 
along  which  the  flow  can  be  increased.  If  the  algorithm  finds  such  a  path,  the  flow 
value  is  increased  appropriately.  If  not,  the  method  iocates  a  cut  whose  capacity 
is  equal  to  the  value  of  the  current  flow. 

Ford  and  Fulkerson’s  method  does  not  automatically  give  a  fast  algorithm 
for  maximum  network  flow.  If  the  capacities  are  large  integers,  the  method  can 
require  enormous  amounts  of  time;  if  the  capacities  are  irrational,  the  method 
need  not  terminate.  However,  if  the  search  for  augmenting  paths  is  systematic, 
the  method  leads  to  a  fast  algorithm.  Table  2  shows  the  running  times  of  various 
maximum  network  flow  algorithms  based  on  this  idea. 

[Table  2] 

Karp  and  Edmonds  [1972]  were  the  first  to  give  a  polynomial-time  algorithm 
for  maximum  flow.  They  showed  that  if  a  shortest  augmenting  path  is  always 
selected,  then  no  more  than  0 (nm)  augmentations  take  place.  From  this  they 
obtained  an  0(mn2)-time  algorithm.  Independently  Dinic  [1970]  made  the  same 
observation,  and  further  noted  that  all  the  augmenting  paths  o?  a  given  length 
can  be  found  at  once,  in  O(nm)  time,  giving  an  overall  bound  of  0(n2m) .  All  the 
recent  progress  on  maximum  flow  is  based  on  Dime’s  work. 

Karzanov  [1974]  improved  Dinic’s  running  time  to  0(n3)  by  discovering  how 
to  find  all  augmenting  paths  of  a  given  length  in  0(n2)  time.  Karsanov’s  algorithm 
is  quite  ''omplicated,  but  Malhotra,  Kumar,  and  Maheshwari  [1978]  obtained  a 
very  simple  algorithm  that  achieves  the  same  time  bound.  Cherkasky  [1977]  dis¬ 
covered  an  0(n2m1/2)  algorithm,  improved  by  Galil  [1978]  to  0(n5/3m2/3).  Galil 
and  Naamad  [1979]  and  independently  Shiloach  [1978]  found  an  0(nm(log  n)2) 
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algorithm,  which  Sleator  and  Tarjan  [1§80]  improved  to  O(imlogn). 

The  Sleator- Tarjan  algorithm  obtains  its  speed  bj  using  sophisticated  data 
structures  that  maintain  the  flow  information  implicitly;  thus  it-  is  not  necessary 
to  perform  an  augmentation  by  changing  the  flow  in  every  edge  of  the  augmenting 
path.  The  general  method  ic  illustrated  in  Figure  4.  The  algorithm  maintains  a 
tree  of  edges  with  residual  capacity  whose  root  is  the  sink.  The  path  from  the 
source  to  the  sink  in  this  tree  defines  an  augmenting  path.  An  augmentation  is 
performed  on  this  path,  saturating  at  least  one  edge  and  causing  the  tree  to  break 
into  at  least  two  edges.  The  tree  is  reassembled  by  adding  new  edges  with  residua! 
capacity,  and  the  process  is  repeated. 

[Figure  4] 

[Figure  5] 

To  represent  the  tree,  the  algorithm  decomposes  it  into  paths,  as  in  Figure 
5.  Before  an  augmentation  is  performed,  the  paths  representing  the  tree  are 
rearranged,  by  splitting  and  concatenation,  so  that  the  source  and  sink  are  on 
the  same  path.  Then  the  augmentation  proceeds.  Galil  and  Naamad  [1979] 
and  Shiloach  [1979]  showed  that  only  O(logn)  splits  and  concatenations  occur 
per  augmentation.  By  representing  each  path  of  the  tree  by  a  data  structure 
consisting  of  a  balanced  binary  tree,  they  obtained  an  Oflog  n)  bound  per  split 
or  concatenation,  an  0((log  n)2)  bound  per  augmentation,  and  an  0(nm(log  n)2) 
bound  overall.  By  representing  each  path  by  a  dynamic  dictionary  implemented 
as  described  in  Section  5,  Sleator  and  Tarjan  were  able  to  reduce  the  time  per 
augmentation  to  O(logn),  saving  a  factor  of  logn  in  the  overall  running  time. 
This  algorithm  seems  hard  to  beat;  further  improvements  in  maximum  flow  may 
require  a  basic  approach  different  from  and  more  powerful  than  Dinic’s. 


7.  Remarks. 

What  are  we  to  conclude  from  all  these  new  results  in  combinatorial  com¬ 
plexity?  First,  it  is  clear  that  NP-completeness  is  a  very  powerful  and  precise 
tool  for  classifying  combinatorial  problems;  it  seems  that  any  natural  problem 
is  either  NP-hard  or  has  a  polynomial-time  algorithm.  The  candidates  for  coun¬ 
terexamples,  such  as  linear  programming  and  graph  isomorphism,  are  yielding  to 
diligent  attack.  Thus  the  P  —  NP  question  becomes,  if  anything,  even  more 
important.  In  general,  it  seems  that  the  lack  of  a  non-trivial  lower  bound  for  a 
problem  is  a  good  reason  to  believe  that  faster  algorithms  exist  for  it. 

Second,  some  polynomial-time  algorithms,  such  as  the  sophisticated  algo¬ 
rithms  for  network  flow,  show  promise  of  being  quite  practical.  Others,  such  as  the 
fastest  methods  for  matrix  multiplication,  are  only  asymptotic  results  and  seem 
to  hold  no  implications  for  practice.  In  order  to  detect  such  differences,  much 
more  study  is  needed  of  algorithmic  overhead,  the  associated  constant  factors, 
and  the  practical  trade-offs  between  algorithms. 

Third,  the  careful  and  systematic  study  of  data  structures  is  extremely  im¬ 
portant  in  the  design  of  algorithms  that  are  fast  both  in  theory  and  in  practice. 
In  particular,  there  is  much  to  be  learned  about  the  properties  of  various  kinds  of 
trees  and  their  use  as  data  structures.  We  still  lack  an  adequate  theory  that  will 
fit  the  appropriate  data  structure  to  each  problem  we  wish  to  solve. 

Fourth,  the  advent  of  very-large-scale  integrated  circuits  has  raised  entirely 
new  questions  for  combinatorial  complexity.  We  are  faced  with  the  problems 
of  designing  new  models  of  complexity  matched  to  the  new  hardware,  and  of 
discovering  what  part  of  the  knowledge  obtained  for  sequential  algorithms  will 
translate  into  the  new  framework,  which  will  incorporate  large-scale  concurrency. 
Thus  parallel  algorithms  and  spatial  layout  problems  are  important  topics  for 
future  research. 
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Figure  U.  Candidate  edges  for  augmenting  paths.  Bold  edges 

denote  spanning  tree.  Sending  flow  along  path  from 
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recomposition  of  the  original  t  •?*-  or  'rvjpu.r«i  U  into 
paths,  one  of  which  leads  from  source  to  sink. 


50 


Date 


Discoverer 


Exponent  of  n 


1969 

Strassen 

2.807 

October 

1973 

Pan 

2.795 

November  1973 

Bini,  et.al. 

2.78 

June 

1979 

Schbnhage 

2.609 

October 

1979 

Pan 

2.605 

October 

1979 

Schbnhage 

2.5U8 

October 

1979 

Pan  and  Winograd 

2.522 

March 

1980 

Pan 

2.1+9  + 

Table  1.  Improvements  in  matrix  multiplication 


Date 

Discoverer 

Running  Time 

1956 

Ford  and  Fulkerson 

1969 

Edmonds  and  Karp 

0(nm2) 

1970 

Dinic 

0(n2m) 

197^ 

Karzanov 

0(1) ?) 

1978 

Malhotra,  et,  al. 

0(tP) 

1977 

Cherkasky 

0(n2  m1//2) 

1970 

Galil 

0(n5/5m2/5) 

1979 

Galil  and  Naamad,  Shiloac'n 

0(n  m(log  n)2} 

1930 

Sleator  and  Tar j an 

O(nmlogn') 

Table  2.  Improvements  in  maximum  network  flow, 


